AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.23145	1.2
	2_DecisionTree	Decision Tree	logloss	1.04184	4.79
the best	3_Default_Xgboost	Xgboost	logloss	0.596477	8.25
	4_Default_NeuralNetwork	Neural Network	logloss	1.04919	1.83
	5_Default_RandomForest	Random Forest	logloss	0.990921	4.12
	Ensemble	Ensemble	logloss	0.596477	0.2

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

0.9 seconds

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0	0	0	0.493324	0	0	0.493324	0.0822207	0.243369	1.23145
recall	0	0	0	1	0	0	0.493324	0.166667	0.493324	1.23145
f1-score	0	0	0	0.660706	0	0	0.493324	0.110118	0.325942	1.23145
support	451	65	822	1404	59	45	0.493324	2846	2846	1.23145

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	0	0	0	451	0	0
Labeled as cloudy	0	0	0	65	0	0
Labeled as partly-cloudy-day	0	0	0	822	0	0
Labeled as rain	0	0	0	1404	0	0
Labeled as snow	0	0	0	59	0	0
Labeled as wind	0	0	0	45	0	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

4.3 seconds

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0.561644	0	0.344008	0.636836	0	0	0.53338	0.257081	0.502528	1.04184
recall	0.181818	0	0.405109	0.785613	0	0	0.53338	0.228757	0.53338	1.04184
f1-score	0.274707	0	0.372067	0.703444	0	0	0.53338	0.225036	0.498021	1.04184
support	451	65	822	1404	59	45	0.53338	2846	2846	1.04184

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	82	0	239	130	0	0
Labeled as cloudy	0	0	57	8	0	0
Labeled as partly-cloudy-day	43	0	333	446	0	0
Labeled as rain	21	0	280	1103	0	0
Labeled as snow	0	0	59	0	0	0
Labeled as wind	0	0	0	45	0	0

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence clear-day (Fold 1)

Dependence cloudy (Fold 1)

Dependence partly-cloudy-day (Fold 1)

Dependence rain (Fold 1)

Dependence snow (Fold 1)

Dependence wind (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

7.7 seconds

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0.663158	0.439024	0.662112	0.798103	0.695652	0.382353	0.723823	0.606734	0.720542	0.596477
recall	0.558758	0.553846	0.648418	0.839031	0.813559	0.288889	0.723823	0.617084	0.723823	0.596477
f1-score	0.606498	0.489796	0.655194	0.818056	0.75	0.329114	0.723823	0.60811	0.720853	0.596477
support	451	65	822	1404	59	45	0.723823	2846	2846	0.596477

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	252	11	112	73	2	1
Labeled as cloudy	3	36	10	14	2	0
Labeled as partly-cloudy-day	65	18	533	186	14	6
Labeled as rain	60	17	132	1178	3	14
Labeled as snow	0	0	6	5	48	0
Labeled as wind	0	0	12	20	0	13

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence clear-day (Fold 1)

Dependence cloudy (Fold 1)

Dependence partly-cloudy-day (Fold 1)

Dependence rain (Fold 1)

Dependence snow (Fold 1)

Dependence wind (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.4 seconds

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0.375	0	0.353478	0.617156	0	0	0.516163	0.224272	0.465977	1.04919
recall	0.192905	0	0.377129	0.763533	0	0	0.516163	0.222261	0.516163	1.04919
f1-score	0.254758	0	0.364921	0.682585	0	0	0.516163	0.217044	0.482505	1.04919
support	451	65	822	1404	59	45	0.516163	2846	2846	1.04919

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	87	0	175	189	0	0
Labeled as cloudy	1	0	41	23	0	0
Labeled as partly-cloudy-day	73	0	310	439	0	0
Labeled as rain	71	0	261	1072	0	0
Labeled as snow	0	0	59	0	0	0
Labeled as wind	0	0	31	14	0	0

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

3.6 seconds

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0.698276	0	0.37193	0.630933	0	0	0.555868	0.283523	0.529332	0.990921
recall	0.179601	0	0.386861	0.842593	0	0	0.555868	0.234842	0.555868	0.990921
f1-score	0.285714	0	0.379249	0.721561	0	0	0.555868	0.231087	0.510777	0.990921
support	451	65	822	1404	59	45	0.555868	2846	2846	0.990921

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	81	0	207	163	0	0
Labeled as cloudy	3	0	46	16	0	0
Labeled as partly-cloudy-day	20	0	318	484	0	0
Labeled as rain	12	0	209	1183	0	0
Labeled as snow	0	0	59	0	0	0
Labeled as wind	0	0	16	29	0	0

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence clear-day (Fold 1)

Dependence cloudy (Fold 1)

Dependence partly-cloudy-day (Fold 1)

Dependence rain (Fold 1)

Dependence snow (Fold 1)

Dependence wind (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	1

Metric details

	clear-day	cloudy	partly-cloudy-day	rain	snow	wind	accuracy	macro avg	weighted avg	logloss
precision	0.663158	0.439024	0.662112	0.798103	0.695652	0.382353	0.723823	0.606734	0.720542	0.596477
recall	0.558758	0.553846	0.648418	0.839031	0.813559	0.288889	0.723823	0.617084	0.723823	0.596477
f1-score	0.606498	0.489796	0.655194	0.818056	0.75	0.329114	0.723823	0.60811	0.720853	0.596477
support	451	65	822	1404	59	45	0.723823	2846	2846	0.596477

Confusion matrix

	Predicted as clear-day	Predicted as cloudy	Predicted as partly-cloudy-day	Predicted as rain	Predicted as snow	Predicted as wind
Labeled as clear-day	252	11	112	73	2	1
Labeled as cloudy	3	36	10	14	2	0
Labeled as partly-cloudy-day	65	18	533	186	14	6
Labeled as rain	60	17	132	1178	3	14
Labeled as snow	0	0	6	5	48	0
Labeled as wind	0	0	12	20	0	13

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back